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ABSTRACT 

Funded in 1999, the "Learning by Doing" (LBD) program of the 
Minnesota State Colleges and Universities system Center for Teaching and 
Learning was designed to increase faculty knowledge, skills, and confidence 
in using active learning strategies. This report marks the end of an in-depth 
evaluation study of active learning and the effects of LBD on two populations 
of full-time faculty, those who were randomly selected form the population at 
large and those who applied for and received an LBD learning grant. The study 
was conducted by an external evaluation team, and the methodology was 
observational rather than experimental. Nearly 1,000 faculty members at large 
and 83 faculty grantees completed a faculty survey abut LBD. More than 3,400 
students from classrooms of faculty at large and grantees completed 1 of 3 
annual waves of the student survey. The study found that the LBD faculty 
development program helped develop a cadre of motivated faculty and 
demonstrated that greater use of active learning teaching strategies can 
increase student effort and involvement with learning. Implications for the 
development of the program are than future faculty development efforts should 
focus on peer . advocates, team grants, and other strategies at the campus 
level, with training and support from the system-wide Center for Teaching and 
Learning. An appendix contains a summary of significant differences between 
grantees and faculty at large on the student survey forms. (SLD) 
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FINAL REPORT SUMMARY 



Evaluation of the Minnesota State Colleges and Universities 
“Learning By Doing” Faculty Development Program 

Active learning is .. . 

“Providing an environment and creating a method that allows students to test or 
exercise the ideas to which they are being exposed. ” 

“When learning is more than memorization; it is relating new knowledge to 
existing knowledge and sharing this understanding with others. " 

“Reality based; working on real problems to help real people. ” 

“When students participate with hands-on activities that elicit critical thinking 
skills and reinforce real-life experiences. ” 

“It is redundant. If the student is not actively participating, learning does not 
take place. Active learning means learning. ” 

These are just some of the ways that faculty members defined the central concept that 
served as the focus of a three-year faculty development effort by the Minnesota State 
Colleges and Universities’ system Center for Teaching and Learning. Funded in 1999 
with a 1 .6 million dollar grant from the Bush Foundation, the “Learning By Doing” 
(LBD) program was designed to increase faculty knowledge, skills, and confidence in 
using active learning teaching strategies. The intent also was to increase faculty 
participation in campus-based and system-wide faculty development activities designed 
to enhance teaching and learning. Last but not least, the LBD program hoped that greater 
use of active learning teaching techniques by faculty would lead to increased student 
satisfaction and involvement in their own learning. 

Learning By Doing Program Activities 

In 1999, the LBD grant was the largest grant the Bush Foundation had made to a single 
educational system; it had followed on the heels of planning grants that Bush had 
awarded during 1996-98. This investment was matched internally with annual 
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contributions of up to $200,000 from the Office of the Chancellor in support of the LBD 
program. Over one million dollars of the implementation grant was passed on to MnSCU 
campuses between 2000 and 2002 in the form of teaching grants to individual faculty 
members, faculty teams, and institutions, in amounts ranging from $1,000 to $25,000. 

Between fall of 1999 and spring of 2002, the LBD program was guided by Center for 
Teaching and Learning (CTL) staff members and a 12-member advisory committee 
comprised of MnSCU faculty, administrators, and other higher education professionals. 

In addition to administering two rounds of teaching grants to 150 recipients, the CTL 
provided summer conferences, traveling workshops, training for campus leaders, 
educational resources, and technical assistance to help faculty integrate active learning 
strategies and instructional technology into their courses. A website was developed to 
serve as a clearing house of information about teaching and learning. By spring of 2002, 
over 800 faculty members had participated in a LBD conference or workshop. 

The Evaluation 

This report marks the end of an in-depth evaluation study of active learning and the 
effects of LBD on two populations of full-time faculty: faculty who were randomly 
selected from the population at large, and faculty who applied for and received a LBD 
teaching grant. The $193,739 study was conducted by an external evaluation team that 
was contracted by MnSCU in 1999 as part of their Bush Foundation proposal. 1 

In the broadest sense, the LBD evaluation sought to understand “what difference” faculty 
development made on faculty members’ teaching practices and attitudes. The evaluation 
also focused on students’ perceptions: how much active learning did they observe in 
classrooms? How did they feel about the teaching methods used in the course? What 
effects did those teaching methods have on them? Underlying the LBD program was the 
assumption that active learning was a proven teaching approach that would increase 

1 Previous interim reports describing the study, as well as the full technical report, are available from the 
principal author. Excerpts and individual reports for subsystems within MnSCU are available from the 
CTL website (www.ctl.mnscu.edu). 
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student involvement and learning. The testing of this assumption became an important 
by-product of the evaluation. 

Background 

“Active learning” was defined broadly by LBD program staff as “anything that involves 
students doing something and thinking about what they are doing.” Rather than 
advocate a particular form of active learning (such as collaborative learning, service 
learning, writing, or technology-assisted strategies), the LBD staff took the position that 
faculty should use whatever strategies best suited their discipline and teaching style in 
order to promote meaningful student engagement. While 42 discrete teaching strategies 
in eight categories were eventually delineated by the evaluation team, faculty members 
were not expected to use all of the strategies, all of the time. Rather, it was hoped that 
faculty would feel proficient in a wide array of strategies, and use the most appropriate 
strategy with confidence, when particular learning objectives called for it. 

While diversity of use was expected, it was hoped that more faculty would use more of 
these teaching strategies, more often, as a result of the LBD program. It was also 
expected that students would observe more of these techniques in grantees’ classrooms, 
compared to classes taught by faculty at large; and that greater use of active learning by 
either grantees or faculty at large would lead to more positive student involvement. 

Methodology 

The methodology used to evaluate LBD was observational, rather than experimental. 

The evaluation team could not randomly award faculty a LBD teaching grant, nor 
randomly assign students to LBD grantee classrooms. What the evaluation team did have 
was access to typical faculty members and their students, as well as grantees’ and their 
students. The study design capitalized on naturally occurring variations in teaching 



2 Bonwell, C., and J. Eison (1991). Active Learning: Creating Excitement in the Classroom, ASHE-ERIC 
Higher Education Report, No. 1 . 
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practices among the faculty at large in 2000 to create two subgroups: “More” and “Less 
Frequent” users of active learning. The evaluation used several sets of comparisons 
between these “More” and “Less Frequent” subgroups, and between the average faculty 
member at large and grantees, to address key evaluation questions. 

Two principal tools — a four-page Faculty Survey and a two-page Student Survey — were 
developed. The surveys were built in parallel: faculty members were asked to report their 
use of the 42 active learning teaching strategies, students were asked to report their 
observation of a similar list of active learning activities in class. 3 Additional items in 
both surveys were developed to capture demographic and other variables. Respondents 
to the Faculty Survey were asked about their satisfaction with teaching, their perceptions 
of active learning, confidence in engaging students, and participation in faculty 
development activities. They were also asked to define active learning in the survey’s 
sole open-ended question. Additional items in the Student Survey asked respondents 
about their satisfaction with the teaching methods used in the course, their attitudes and 
behaviors during class, hours spent outside of class studying, class attendance, and other 
outcomes. 4 

These two surveys were administered to a random sample of faculty at large and students 
in their classrooms, and to all lead faculty with LBD grants and their students. Nearly 
1,000 faculty members at large and 83 faculty grantees completed the Faculty Survey. 
Over 3,400 students from both faculty at large and grantee classrooms completed one of 
three annual waves of the Student Survey. 5 



3 Active learning strategies were grouped into eight thematic categories: Lecture, Discussion, Writing, 
Technology-Assisted, Interactive, Collaborative Learning, Assessment, and Experiential. Items comprising 
these scales in the Faculty Survey and Student Survey were found to have moderate to strong reliability. 

4 Several items were drawn from the National Survey of Student Engagement. The NSSE is a product of 
the National Center for Higher Education Management Systems and the Indiana University Center for 
Postsecondary Research and Planning, with support from the Pew Charitable Trusts. The NSSE is a 4-page 
machine scored instrument covered college activities, the college environment, student estimated “gains,” 
and opinions about their institution. 

s Response rates to the Faculty Survey for faculty at large were 55% (2000) and 43% (2002); for faculty 
grantees it was 86% (2001) and 61% (2002). Participation by faculty at large in the Student Survey fell 
significantly over the 3 years, from n = 66 (100%) in 2000 to n = 35 (53%) in 2002. Participation by 
faculty grantees in the Student Survey ranged from 42% to 63%. 



Professional Evaluation Services / Professional Data Analysts, Inc. 



6 



Key Findings for Faculty at Large 6 

• Use of active learning teaching strategies was fairly high at baseline. 

The baseline usage of active learning teaching methods turned out to be rather high. 7 
Faculty at large reported using a fairly high number of strategies (about 20 out of 42, 
on average). Even faculty who were grouped into the “Less Frequent” subgroup 
reported using an average of 15 strategies “sometimes,” if not “frequently.” The 
validity of faculty members’ self reports was supported by student observations in a 
subset of classrooms. 8 

• Short-term, less innovative strategies were most common. 

The typical faculty member favored eight, fairly traditional active learning strategies, 
such as basic student assessment 9 and other short-term strategies that could be 
embedded in a lecture or discussion course. About half of the 42 strategies listed 
were never or rarely used. Additionally, the repertoire of methods was more limited 
for some faculty members than others. Greatest use of active learning strategies was 
occurring in the technical colleges, in humanities courses, and by mid-career faculty. 
Active learning strategies were least used by faculty in the early or late part of their 
careers, by those in state universities, and by science faculty. 

• Faculty agreed on the meaning of active learning, broadly defined. 

Faculty agreed on a broad definition of the term, with a large majority using phrases 
that echoed those being promoted by the LBD staff (“active vs. passive participation 
in learning”), or the name of the initiative itself (“learning by doing”). These 
definitions were “correct,” but sometimes limited. The better definitions stressed the 
underlying cognitive engagement of students in thinking about what they were 
doing — they did not just equate active learning with “doing something,” or a 

6 See the Appendix for complete list of study questions and results. 

7 These results are based on a 56% response rate to the Faculty Survey in 2000 (n = 555). 

8 The average correlation between 42 items measuring active learning on the Faculty Survey, and 32 items 

measuring the same strategies on the Student Survey, was r = .45 (si! = .10). 
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